1. Field of the Invention
The present invention relates to a picture transmission system, picture coder, and picture decoder, and more particularly to the prevention of picture quality degradation caused by frame dropouts or frame skipping.
2. Description of the Prior Art
Recently, there has been a proliferation of transmission systems that transmit picture signals over networks: examples include videotelephone, videoconferencing, and video-on-demand (VOD). To keep pace with the technical advances, standardization of the picture coding method is now in progress.
The picture coding method is classified into two: one is a coding method that employs both intra-frame coding and inter-frame coding and the other is a coding method that employs only intra-frame coding.
The method that employs both intra- and inter-frame coding has been used as a moving-picture communication/accumulation coding method proposed by the ITU-T Recommendation H.261 and Moving Picture Experts Group (MPEG). FIG. 2 is an example of frames arranged in time sequence. The coding method adopted by the ITU-T Recommendation H.261, shown in FIG. 2, performs intra-frame coding (I frames a and i) at regular intervals and, at other times, performs inter-frame coding for each of P frames (inter-frame coded frames, b-h, j-k) with reference to the immediately preceding frame to remove temporal redundancy. In the following discussion, a frame coded in the intra-frame coding method is referred to as an I frame, while a frame coded in the inter-frame coding method is referred to as a P frame.
On the other hand, the method that employs only intra-frame coding has been used as a still picture coding method such as that adopted by Joint Photographic Coding Experts Group (JPEG). This method performs intra-frame coding on all the frames, as shown in FIG. 3.
The method proposed by Recommendation H.261, which performs the inter-frame coding of each frame by referencing the immediately preceding frame, requires that all the frames be transmitted in the correct sequence. For a telephone line or an ISDN line over which data is transmitted after a connection with a partner is established, data reaches the partner without loss and in the correct sequence. However, for an Ethernet LAN or an ATM network in which data is divided into small units (called packets or cells) before transmission, there is a possibility that packets are lost or transmitted in an incorrect sequence.
In general, networks employ a protocol (for example, TCP Transmission Control Protocol) in which the transmitting device sends packets with attached serial numbers, and the receiving device rearranges the packets in the correct sequence, confirms their arrival, and sends requests for the retransmission of non-arriving packets back to the transmitting device in order to deal with these problems and increase network reliability.
However, when network operation is unstable and packets are dropped frequently, retransmission under this type of protocol can cause large cumulative delays to build up, which is inappropriate for the real-time transmission of moving pictures. In some cases, it is preferable to display new data, even if skipping of a frame, rather than retransmitting old data, especially when new data can be displayed immediately.
Broadcasting and multicasting are schemes which send data to a plurality of sites at one time. However, when packet dropout occurs during transmission of a packet to one of the sites, the above protocol requires that the same packet be sent even to those sites which have received the packet successfully, significantly increasing the network load. Broadcasting and multicasting are therefore performed using a protocol that does not re-transmit a packet, such as the User Datagram Protocol (UDP); as a result, the probability of packet dropout increases.
In wireless networks, the data error rate or data drop-out rate is high not only in packet transmission but also in established line transmission due to fading and so forth. In addition, when the errors exceed the error-correcting capability of the receiving device, some pieces of data are sometimes discarded to receive other pieces of data successfully. Data dropouts in wireless networks therefore tend to be larger than in wire networks.
Another problem is that the processing speed of the sending device is not always equal to that of the receiving device. For example, decoding all the frames on a slower receiving device would put much frame data in the wait state, causing long delays. This requires the receiving device to intentionally skip frames. However, when there is no decoding data for a frame preceding the current frame, as in the inter-frame coding method according to the prior art, the current frame cannot be decoded. Hence, this prevents free skipping of frames.
FIG. 4 shows an example of frame dropout that may occur during transmission of frames according to the protocol proposed by Recommendation H.261. When frame e is dropped out or cannot be decoded because of slow processing, the P frames (f, g, h) cannot be decoded until the next I frame, i, is received.
Thus, in order to send all the frames successfully in a network in which frame dropout or frame skipping occurs frequently, intra-frame coding such as JPEG is performed, instead of inter-frame coding, for all the frames. For example, even when frame e is dropped out in the JPEG coding shown in FIG. 5, the next and the following frames may be decoded successfully. However, the problems with this method, which does not use inter-frame coding, are that there is temporal redundancy, whereby coding and decoding are inefficient, thereby leading to necessity of transmitting a large amount of data.
Thus, there has been a desire for a coder and decoder which decodes P frames without having to wait for the next I frame and which prevents coding efficiency from being decreased even in an environment in which there is a possibility that frames are dropped out or skipped.